AMENDMENT AND RESPONSE UNDER 37 CFR § 1.116 Page 2 

Serial Number: 09/636,000 Dkt: BU2446/0033-033001 

Filing Date: August 9, 2000 

Title: Maximum Likelihood Sequence Estimator which Computes Branch Metrics in Real Time 

IN THE CLAIMS 

Please amend the claims as follows: 

1 . (Currently Amended) A Maximum likelihood sequence estimator (MLSE) for 
estimating a possible sequence of transmitted symbols received over a dispersive communication 
channel, wherein a trellis of states and trellis paths are associated with the possible sequence of 
transmitted s ymbol symbols s e qu e nc e, said MLSE comprising: 

a plurality of data sources relating respectively to state transition probabilities and 
observed values of received data symbols; 

means for calculating and storing the-ajikelihood metric and_a survivor bit for each state 
of the trellis using values from said plurality of data sources, comprising means for computing 
branch metrics for the trellis by setting a first portion of the branch metrics equal to a coefficient 
which models an autocorrelation of an impulse response of a channel and setting a second 
portion of the branch metrics equal to a sum or difference of a prior value of a branch metric and 
a coefficient which models an autocorrelation of an impulse response of a channel; 

means for determination of the-afinal state on the-a_maximum likelihood path in the 
trellis; and 

means for calculating the-a_maximum likelihood sequence of transmitted symbols in a 
backward trace through the trellis using said stored survivor bits. 

2. (Currently Amended) The MLSE of claim 1 further comprising means for 
computing supporting branch metric paramet e r parameters, calculations wherein said branch 
metric parameters are computed recursively for a Gray coded sequence of states, wherein said 
recursive computation requires only a single addition operation per branch metric parameter per 
state, thereby substantially reducing the-anumber of computational steps required per branch 
metric parameter calculation. 

3. (Currently Amended) The MLSE of Claim 2 wherein said branch metric 
parameters are pre-computed and stored in a_data memory prior to_a forward trace through the 
trellis, wherein said stored branch metric parameters are retrieved from said data memory as 
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needed to support state metric calculations subsequently performed in said forward trace through 
the trellis. 

4. (Currently Amended) The MLSE of Claim 2 wherein said branch metric 
parameters are computed in real time as needed for state metric calculations, and wherein the 
sequencing of the- said states is performed according to a Gray code for both branch metric 
calculations and state metric calculations, thereby achieving a substantial savings in adata 
storage requirement. 

5. (Currently Amended) The MLSE of Claim 1 further including means to utilize 
prior knowledge about the initial trellis state to enhance MLSE estimation performance of said 
transmitted sequence, said performance enhancing means including: 

anjnitial state register paired with an_initial state mask r e gister register, wherein a- said 
pair of registers define a set of valid initial states representing prior knowledge about the 
transmitted sequence; and 

means for initialization of trellis state metrics such that the-an MLSE Viterbi algorithm 
selection of the maximum likelihood path in the trellis is confined only to paths having a valid 
initial state. 

6. (Currently Amended) A Maximum likelihood sequence estimator (MLSE) for 
estimating a possible sequence of transmitted symbols received over a dispersive communication 
channel, wherein a trellis of states and trellis paths are associated with the possible sequence of 
transmitted symbol symbols s e quenc e, said MLSE comprising: 

a plurality of data sources relating respectively to state transition probabilities and 
observed values of received data symbols; 

means for calculating and storing the-ahkelihood metric and a_survivor bit for each state 
of the trellis using values from said data sources; 

means for determination of the-a_final state on the-a_maximum likelihood path in the 

trellis; 
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means for calculating the-amaximum likelihood sequence of transmitted symbols in a 
backward trace through the trellis using said stored survivor bits; and 

means to utilize prior knowledge about the-an initial trellis state to enhance MLSE 
estimation performance of said transmitted sequence, said performance enhancing means 
including: 

affinal state register paired with afinal state mask r e gist e r register, wherein said 
register pair define a set of valid final states representing prior knowledge about the 
transmitted sequence; and 

means for selection of the-a_final state of the trellis on the maximum likelihood 
path such that the-an MLSE Viterbi algorithm selection of the maximum likelihood path 
in the trellis is confined only to paths having a valid final state, 

7. (Previously Presented) The MLSE of Claim 2 further including means to provide 
sufficient data for a class of soft decision generators that are dependent only on partial path 
metrics. 

8. (Currently Amended) A method of computing a maximum likelihood sequence 
estimate comprising: 

providing an initial state; 

providing an initial state mask comprising a plurality of bits having either a first polarity 
or a second plurality polarity ; and 

determining a plurality of valid initial states by: 
starting with the initial state; and 

substituting a don't care for each bit in the initial state which has a corresponding 
bit having a first state in the initial state mask, 

wherein the valid initial states are defined by either a one or a zero in the bit 
position having a don't care, and the same bit as the initial state in the other positions. 



9. (Currently Amended) The method of claim 8 wherein the initial state mask is 
determined by power up charact e ristic characteristics of a transmitter. 
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10. (Original) The method of claim 9 wherein the transmitter is complaint with the 
Global System for Mobile Communications standard. 

1 1 . (Previously Presented) A method of computing a maximum likelihood sequence 
estimate comprising: 

providing a trellis comprising a plurality of nodes corresponding to a plurality of states at 
a plurality of stages; and 

computing branch metrics for the trellis by setting a first portion of the branch metrics 
equal to a coefficient which models an autocorrelation of an impulse response of a channel and 
setting a second portion of the branch metrics equal to a sum or difference of a prior value of a 
branch metric and a coefficient which models an autocorrelation of an impulse response of a 
channel. 

12. (Previously Presented) The method of claim 1 1 wherein a present state is 
incremented to a next state by changing only one bit in accordance with a Gray code. 

13. (Currently Amended) The MLSE of Claim 6 comprising a comparator configured 
to test for equality of an output of the-a_final state register and a current state in accordance with 
an output of the final state mask register. 

14. (Currently Amended) The MLSE of Claim 13 comprising a multiplexer for 
selecting the current state or a prior state in accordance with the test for equality . 

15. (Currently Amended) The MLSE of Claim 14 comprising a latch for storing the 
current state in accordance with the tes t for equality . 



16. (Currently Amended) The MLSE of Claim 6 comprising means to utilize prior 
knowledge about the initial trellis state to enhance MLSE estimation performance of said 
transmitted sequence, said performance enhancing means comprising: 
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anjnitial state register paired with anjnitial state mask r e gist e r register, wherein a- said 
pair of registers define a set of valid initial states representing prior knowledge about the 
transmitted sequence; and 

means for initialization of trellis state metrics such that the MLSE Viterbi algorithm 
selection of the maximum likelihood path in the trellis is confined only to paths having a valid 
initial state. 

17. (Previously Presented) The method of claim 8 comprising testing for equality of 
the initial state and a current state in accordance with the initial state mask. 

18. (Currently Amended) The method of claim 17 comprising controlling a 
multiplexer in accordance with the testing for equality of the initial state to select an initial 
metric value associated with a valid initial state or an initial metric value associated with an 
invalid initial state. 

19. (Previously Presented) The method of claim 8 comprising associating initial 
metric values with the valid initial states and invalid initial states, wherein the initial metric 
values associated with the valid initial states are greater in magnitude than the initial metric 
values associated with the invalid initial states. 

20. (Previously Presented) The method of claim 19 wherein the initial metric values 
associated with the invalid initial states are negative values. 

21. (Previously Presented) The method of claim 8 comprising: 
providing final state data; 

providing a final state mask; 

determining a plurality of valid final states in accordance with the final state data and the 
final state mask; and 

selecting path matrices corresponding to a valid initial state and a valid final state. 
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22-30. (Canceled) 



